When the Map Is Better Than the Territory

The causal structure of any system can be analyzed at a multitude of spatial and temporal scales. It has long been thought that while higher scale (macro) descriptions of causal structure may be useful to observers, they are at best a compressed description and at worse leave out critical information. However, recent research applying information theory to causal analysis has shown that the causal structure of some systems can actually come into focus (be more informative) at a macroscale (Hoel et al. 2013). That is, a macro model of a system (a map) can be more informative than a fully detailed model of the system (the territory). This has been called “causal emergence.” While causal emergence may at first glance seem counterintuitive, this paper grounds the phenomenon in a classic concept from information theory: Shannon’s discovery of the channel capacity. I argue that systems have a particular causal capacity, and that different causal models of those systems take advantage of that capacity to various degrees. For some systems, only macroscale causal models use the full causal capacity. Such macroscale causal models can either be coarse-grains, or may leave variables and states out of the model (exogenous) in various ways, which can improve the model’s efficacy and its informativeness via the same mathematical principles of how error-correcting codes take advantage of an information channel’s capacity. As model choice increase, the causal capacity of a system approaches the channel capacity. Ultimately, this provides a general framework for understanding how the causal structure of some systems cannot be fully captured by even the most detailed microscopic model. Introduction. It has been difficult to find a causal role for the macroscales of a system (Fodor 1974; Kim 2000). But a way forward is to consider this issue not as a philosophical problem but rather as a problem of causal model choice. Causal models are those that represent the influence of subparts of a system on other subparts, or over the system as a whole. A causal model may represent state transitions, like Markov chains, or may represent the influence or connectivity of elements, such as circuit diagrams, directed graphs (also called causal Bayesian networks), networks of interconnected mechanisms, or neuron diagrams. Using causal models in the form of networks of logic gates, actual causal emergence was first demonstrated in Hoel et al. (2013). Causal emergence is when the macro beats the micro in terms of the efficacy, informativeness, or power of its causal relationships. It is identified by comparing macroscales to their underlying microscales, and analyzing the causal structure of both using information theory. Here it is revealed that causal emergence is related to a classic concept in information theory, Shannon’s channel capacity (Shannon 1949), thus grounding it rigorously in another well-known mathematical phenomenon. There is a natural, but unremarked upon, connection between causation and information theory. Both causation and information are defined in respect to the nonexistent: causation relies on counterfactual statements, and information theory on unsent signals. While their similarities have not been enumerated explicitly, it is unsurprising that a combination of the two has begun to gain traction. Causal analysis is performed on a set of interacting elements or state transitions, i.e., a causal model, and the hope is that information theory is the best way to quantify and formalize these interactions and transitions. Some information-theoretic constructs already use quasi-causal language; measurements like granger causality (Granger 1969), directed information (Massey 1990), and transfer entropy (Schreiber 2000). Despite being interesting and useful measures, these either fail to accurately measure causation, or are too vague and underdeveloped (Janzing et al. 2013). More explicit attempts to tie information theory to causation use Judea Pearl’s causal calculus (Pearl When the map is better than the territory /